{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: using VMLS.rms in module Main conflicts with an existing identifier.\n"
     ]
    }
   ],
   "source": [
    "using LinearAlgebra\n",
    "using VMLS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 3\n",
    "# Norm and distance\n",
    "### 3.1 Norm\n",
    "**Norm.** The norm $‖x‖$ is written in Julia as `norm(x)`. (It can be evaluated several other ways, too.) The `norm` function is contained in the Julia package `LinearAlgebra`, so you must install and then add this package to use it; see page ix. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3.0, 3.0, 3.0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [ 2, -1, 2 ];\n",
    "norm(x), sqrt(x'*x), sqrt(sum(x.^2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Triangle inequality.** Let’s check the triangle inequality, $‖x+ y‖ ≤ ‖x‖+ ‖y‖$, for some specific values of $x$ and $y$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4.7705830402645715, 5.975354579294137)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = randn(10); y = randn(10);\n",
    "lhs = norm(x+y)\n",
    "rhs = norm(x) + norm(y)\n",
    "lhs,rhs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**RMS value.** The RMS value of a vector $x$ is `rms(x)` $= \\frac{‖x‖}{\\sqrt{n}}$. In Julia, this is expressed as `norm(x)/sqrt(length(x))`. (The VMLS package contains this function, so you can use it once you’ve installed this package.)\n",
    "\n",
    "Let’s define a vector (which represents a signal, i.e., the value of some quantity at uniform space time instances), and find its RMS value. The following code plots the signal, its average value, and two constant signals at `avg(x) ± rms(x)` (Figure [3.1](https://web.stanford.edu/~boyd/vmls/vmls.pdf#figure.3.1))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.04252943783238692"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rms(x) = norm(x) / sqrt(length(x));\n",
    "t = 0:0.01:1; # List of times\n",
    "x = cos.(8*t) - 2*sin.(11*t);\n",
    "avg(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0837556422598"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rms(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 2400 1600\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip9600\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"2400\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip9600)\" points=\"\n",
       "0,1600 2400,1600 2400,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip9601\">\n",
       "    <rect x=\"480\" y=\"0\" width=\"1681\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip9600)\" points=\"\n",
       "141.865,1487.47 2352.76,1487.47 2352.76,47.2441 141.865,47.2441 \n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip9602\">\n",
       "    <rect x=\"141\" y=\"47\" width=\"2212\" height=\"1441\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  204.437,1487.47 204.437,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  725.874,1487.47 725.874,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1247.31,1487.47 1247.31,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1768.75,1487.47 1768.75,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  2290.18,1487.47 2290.18,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  141.865,1187.55 2352.76,1187.55 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  141.865,819.66 2352.76,819.66 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  141.865,451.768 2352.76,451.768 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  141.865,83.8768 2352.76,83.8768 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,1487.47 2352.76,1487.47 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,1487.47 141.865,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  204.437,1487.47 204.437,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  725.874,1487.47 725.874,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1247.31,1487.47 1247.31,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1768.75,1487.47 1768.75,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  2290.18,1487.47 2290.18,1465.87 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,1187.55 175.028,1187.55 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,819.66 175.028,819.66 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,451.768 175.028,451.768 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  141.865,83.8768 175.028,83.8768 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 204.437, 1541.47)\" x=\"204.437\" y=\"1541.47\">0.00</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 725.874, 1541.47)\" x=\"725.874\" y=\"1541.47\">0.25</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1247.31, 1541.47)\" x=\"1247.31\" y=\"1541.47\">0.50</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 1768.75, 1541.47)\" x=\"1768.75\" y=\"1541.47\">0.75</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 2290.18, 1541.47)\" x=\"2290.18\" y=\"1541.47\">1.00</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 117.865, 1205.05)\" x=\"117.865\" y=\"1205.05\">-1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 117.865, 837.16)\" x=\"117.865\" y=\"837.16\">0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 117.865, 469.268)\" x=\"117.865\" y=\"469.268\">1</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip9600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 117.865, 101.377)\" x=\"117.865\" y=\"101.377\">2</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#009af9; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  204.437,451.768 225.294,533.718 246.152,617.037 267.009,700.738 287.867,783.843 308.724,865.394 329.582,944.463 350.439,1020.17 371.297,1091.67 392.154,1158.21 \n",
       "  413.012,1219.08 433.869,1273.67 454.727,1321.43 475.584,1361.93 496.441,1394.81 517.299,1419.83 538.156,1436.83 559.014,1445.78 579.871,1446.71 600.729,1439.8 \n",
       "  621.586,1425.28 642.444,1403.5 663.301,1374.9 684.159,1339.97 705.016,1299.32 725.874,1253.58 746.731,1203.46 767.589,1149.73 788.446,1093.18 809.303,1034.61 \n",
       "  830.161,974.874 851.018,914.805 871.876,855.238 892.733,796.99 913.591,740.852 934.448,687.577 955.306,637.871 976.163,592.385 997.021,551.706 1017.88,516.349 \n",
       "  1038.74,486.751 1059.59,463.268 1080.45,446.167 1101.31,435.624 1122.17,431.724 1143.02,434.46 1163.88,443.729 1184.74,459.34 1205.6,481.013 1226.45,508.382 \n",
       "  1247.31,541.005 1268.17,578.365 1289.03,619.879 1309.88,664.907 1330.74,712.758 1351.6,762.702 1372.46,813.978 1393.31,865.805 1414.17,917.391 1435.03,967.948 \n",
       "  1455.88,1016.7 1476.74,1062.88 1497.6,1105.77 1518.46,1144.7 1539.31,1179.02 1560.17,1208.16 1581.03,1231.63 1601.89,1249 1622.74,1259.91 1643.6,1264.1 \n",
       "  1664.46,1261.41 1685.32,1251.76 1706.17,1235.16 1727.03,1211.72 1747.89,1181.66 1768.75,1145.26 1789.6,1102.91 1810.46,1055.09 1831.32,1002.35 1852.18,945.303 \n",
       "  1873.03,884.648 1893.89,821.127 1914.75,755.536 1935.61,688.705 1956.46,621.496 1977.32,554.785 1998.18,489.455 2019.04,426.385 2039.89,366.437 2060.75,310.445 \n",
       "  2081.61,259.205 2102.47,213.467 2123.32,173.919 2144.18,141.184 2165.04,115.809 2185.9,98.2544 2206.75,88.8944 2227.61,88.0053 2248.47,95.7639 2269.33,112.243 \n",
       "  2290.18,137.412 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#e26f46; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  204.437,835.306 225.294,835.306 246.152,835.306 267.009,835.306 287.867,835.306 308.724,835.306 329.582,835.306 350.439,835.306 371.297,835.306 392.154,835.306 \n",
       "  413.012,835.306 433.869,835.306 454.727,835.306 475.584,835.306 496.441,835.306 517.299,835.306 538.156,835.306 559.014,835.306 579.871,835.306 600.729,835.306 \n",
       "  621.586,835.306 642.444,835.306 663.301,835.306 684.159,835.306 705.016,835.306 725.874,835.306 746.731,835.306 767.589,835.306 788.446,835.306 809.303,835.306 \n",
       "  830.161,835.306 851.018,835.306 871.876,835.306 892.733,835.306 913.591,835.306 934.448,835.306 955.306,835.306 976.163,835.306 997.021,835.306 1017.88,835.306 \n",
       "  1038.74,835.306 1059.59,835.306 1080.45,835.306 1101.31,835.306 1122.17,835.306 1143.02,835.306 1163.88,835.306 1184.74,835.306 1205.6,835.306 1226.45,835.306 \n",
       "  1247.31,835.306 1268.17,835.306 1289.03,835.306 1309.88,835.306 1330.74,835.306 1351.6,835.306 1372.46,835.306 1393.31,835.306 1414.17,835.306 1435.03,835.306 \n",
       "  1455.88,835.306 1476.74,835.306 1497.6,835.306 1518.46,835.306 1539.31,835.306 1560.17,835.306 1581.03,835.306 1601.89,835.306 1622.74,835.306 1643.6,835.306 \n",
       "  1664.46,835.306 1685.32,835.306 1706.17,835.306 1727.03,835.306 1747.89,835.306 1768.75,835.306 1789.6,835.306 1810.46,835.306 1831.32,835.306 1852.18,835.306 \n",
       "  1873.03,835.306 1893.89,835.306 1914.75,835.306 1935.61,835.306 1956.46,835.306 1977.32,835.306 1998.18,835.306 2019.04,835.306 2039.89,835.306 2060.75,835.306 \n",
       "  2081.61,835.306 2102.47,835.306 2123.32,835.306 2144.18,835.306 2165.04,835.306 2185.9,835.306 2206.75,835.306 2227.61,835.306 2248.47,835.306 2269.33,835.306 \n",
       "  2290.18,835.306 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#008000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  204.437,436.602 225.294,436.602 246.152,436.602 267.009,436.602 287.867,436.602 308.724,436.602 329.582,436.602 350.439,436.602 371.297,436.602 392.154,436.602 \n",
       "  413.012,436.602 433.869,436.602 454.727,436.602 475.584,436.602 496.441,436.602 517.299,436.602 538.156,436.602 559.014,436.602 579.871,436.602 600.729,436.602 \n",
       "  621.586,436.602 642.444,436.602 663.301,436.602 684.159,436.602 705.016,436.602 725.874,436.602 746.731,436.602 767.589,436.602 788.446,436.602 809.303,436.602 \n",
       "  830.161,436.602 851.018,436.602 871.876,436.602 892.733,436.602 913.591,436.602 934.448,436.602 955.306,436.602 976.163,436.602 997.021,436.602 1017.88,436.602 \n",
       "  1038.74,436.602 1059.59,436.602 1080.45,436.602 1101.31,436.602 1122.17,436.602 1143.02,436.602 1163.88,436.602 1184.74,436.602 1205.6,436.602 1226.45,436.602 \n",
       "  1247.31,436.602 1268.17,436.602 1289.03,436.602 1309.88,436.602 1330.74,436.602 1351.6,436.602 1372.46,436.602 1393.31,436.602 1414.17,436.602 1435.03,436.602 \n",
       "  1455.88,436.602 1476.74,436.602 1497.6,436.602 1518.46,436.602 1539.31,436.602 1560.17,436.602 1581.03,436.602 1601.89,436.602 1622.74,436.602 1643.6,436.602 \n",
       "  1664.46,436.602 1685.32,436.602 1706.17,436.602 1727.03,436.602 1747.89,436.602 1768.75,436.602 1789.6,436.602 1810.46,436.602 1831.32,436.602 1852.18,436.602 \n",
       "  1873.03,436.602 1893.89,436.602 1914.75,436.602 1935.61,436.602 1956.46,436.602 1977.32,436.602 1998.18,436.602 2019.04,436.602 2039.89,436.602 2060.75,436.602 \n",
       "  2081.61,436.602 2102.47,436.602 2123.32,436.602 2144.18,436.602 2165.04,436.602 2185.9,436.602 2206.75,436.602 2227.61,436.602 2248.47,436.602 2269.33,436.602 \n",
       "  2290.18,436.602 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip9602)\" style=\"stroke:#008000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  204.437,1234.01 225.294,1234.01 246.152,1234.01 267.009,1234.01 287.867,1234.01 308.724,1234.01 329.582,1234.01 350.439,1234.01 371.297,1234.01 392.154,1234.01 \n",
       "  413.012,1234.01 433.869,1234.01 454.727,1234.01 475.584,1234.01 496.441,1234.01 517.299,1234.01 538.156,1234.01 559.014,1234.01 579.871,1234.01 600.729,1234.01 \n",
       "  621.586,1234.01 642.444,1234.01 663.301,1234.01 684.159,1234.01 705.016,1234.01 725.874,1234.01 746.731,1234.01 767.589,1234.01 788.446,1234.01 809.303,1234.01 \n",
       "  830.161,1234.01 851.018,1234.01 871.876,1234.01 892.733,1234.01 913.591,1234.01 934.448,1234.01 955.306,1234.01 976.163,1234.01 997.021,1234.01 1017.88,1234.01 \n",
       "  1038.74,1234.01 1059.59,1234.01 1080.45,1234.01 1101.31,1234.01 1122.17,1234.01 1143.02,1234.01 1163.88,1234.01 1184.74,1234.01 1205.6,1234.01 1226.45,1234.01 \n",
       "  1247.31,1234.01 1268.17,1234.01 1289.03,1234.01 1309.88,1234.01 1330.74,1234.01 1351.6,1234.01 1372.46,1234.01 1393.31,1234.01 1414.17,1234.01 1435.03,1234.01 \n",
       "  1455.88,1234.01 1476.74,1234.01 1497.6,1234.01 1518.46,1234.01 1539.31,1234.01 1560.17,1234.01 1581.03,1234.01 1601.89,1234.01 1622.74,1234.01 1643.6,1234.01 \n",
       "  1664.46,1234.01 1685.32,1234.01 1706.17,1234.01 1727.03,1234.01 1747.89,1234.01 1768.75,1234.01 1789.6,1234.01 1810.46,1234.01 1831.32,1234.01 1852.18,1234.01 \n",
       "  1873.03,1234.01 1893.89,1234.01 1914.75,1234.01 1935.61,1234.01 1956.46,1234.01 1977.32,1234.01 1998.18,1234.01 2019.04,1234.01 2039.89,1234.01 2060.75,1234.01 \n",
       "  2081.61,1234.01 2102.47,1234.01 2123.32,1234.01 2144.18,1234.01 2165.04,1234.01 2185.9,1234.01 2206.75,1234.01 2227.61,1234.01 2248.47,1234.01 2269.33,1234.01 \n",
       "  2290.18,1234.01 \n",
       "  \"/>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Plots\n",
    "plot(t, x)\n",
    "plot!(t, avg(x)*ones(length(x)))\n",
    "plot!(t, (avg(x)+rms(x))*ones(length(x)), color = :green)\n",
    "plot!(t, (avg(x)-rms(x))*ones(length(x)), color = :green)\n",
    "plot!(legend = false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Figure 3.1** A signal $x$. The horizontal lines show $avg(x) + rms(x)$, $avg(x)$ and $avg(x)− rms(x)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Chebyshev inequality.** The Chebyshev inequality states that the number of entries of an $n$-vector $x$ that have absolute value at least $a$ is no more than $\\frac{‖x‖^2}{a^2} = n rms(x)^2/a^2$. If this number is, say, $12.15$, we can conclude that no more that $12$ entries have absolute value at least $a$, since the number of entries is an integer. So the Chebyshev bound can be improved to be $floor(‖x‖^2/a)$, where $floor(u)$ is the integer part of a positive number. Let’s define a function with the Chebyshev bound, including the floor function improvement, and apply the bound to the signal found above, for a specific value of $a$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(\"Cheb bound:\", 79.0)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define Chebyshev bound function\n",
    "cheb_bound(x,a) = floor(norm(x)^2/a);\n",
    "a = 1.5;\n",
    "\"Cheb bound:\", cheb_bound(x,a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(\"Number of entries of x with |x_i| >= a:\", 20)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"Number of entries of x with |x_i| >= a:\", sum(abs.(x) .>= a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the last line, the expression `abs.(x) .>= a` creates an array with entries that are Boolean, i.e., `true` or `false`, depending on whether the corresponding entry of `x` satisfies the inequality. When we sum the vector of Booleans, they are automatically converted to (re-cast as) the numbers `1` and `0`, respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Distance\n",
    "\n",
    "**Distance.** The distance between two vectors is $dist(x, y) = ‖x − y‖$. This is written in Julia as `norm(x-y)`. Let’s find the distance between the pairs of the three vectors $u, v,$ and $w$ from page 49 of VMLS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8.36779540858881, 0.3872983346207417, 8.532877591996735)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u = [1.8, 2.0, -3.7, 4.7];\n",
    "v = [0.6, 2.1, 1.9, -1.4];\n",
    "w = [2.0, 1.9, -4.0, 4.6];\n",
    "norm(u-v), norm(u-w), norm(v-w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that $u$ and $w$ are much closer to each other than $u$ and $v$, or $v$ and $w$.\n",
    "\n",
    "**Nearest neighbor.** We define a function that calculates the nearest neighbor of a vector in a list of vectors, and try it on the points in Figure [3.3](https://web.stanford.edu/~boyd/vmls/vmls.pdf#figure.3.3) of VMLS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{Float64,1}:\n",
       " 5.5\n",
       " 4.0"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nearest_neighbor(x,z) = z[ argmin([norm(x-y) for y in z]) ];\n",
    "z = ( [2,1], [7,2], [5.5,4], [4,8], [1,5], [9,6] );\n",
    "nearest_neighbor([5,6], z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{Int64,1}:\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nearest_neighbor([3,3], z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the first line, the expression `[norm(x-y) for y in z]` uses a convenient construction in Julia. Here `z` is a list of vectors, and the expression expands to an array with elements `norm(x-z[1])`, `norm(x-z[2]), . . . .` The function `argmin` applied to this array returns the index of the smallest element. \n",
    "\n",
    "**De-meaning a vector.** We refer to the vector `x − avg(x)1` as the de-meaned version of `x`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "de_mean(x) = x .- avg(x); # Define de-mean function\n",
    "x = [1, -2.2, 3];\n",
    "avg(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1.4802973661668753e-16"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_tilde = de_mean(x)\n",
    "avg(x_tilde)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(The mean of $x̃$ is very very close to zero.)\n",
    "\n",
    "### 3.3 Standard deviation\n",
    "**Standard deviation.** We can define a function that corresponds to the VMLS definition of the standard deviation of a vector, $std(x) =$ $\\frac{‖x − avg(x)1‖}{\\sqrt{n}}$, where $n$ is the length of the vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2929673241257378"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = rand(100);\n",
    "# VMLS definition of std\n",
    "stdev(x) = norm(x.-avg(x))/sqrt(length(x));\n",
    "stdev(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function is in the VMLS package, so you can use it once you’ve installed this package. (Julia’s Statistics package has a similar function, `std(x)`, which computes the value $\\frac{‖x − avg(x)1‖}{\\sqrt{n − 1}}$, where $n$ is the length of $x$.) \n",
    "\n",
    "**Return and risk.** We evaluate the mean return and risk (measured by standard deviation) of the four time series Figure [3.4](https://web.stanford.edu/~boyd/vmls/vmls.pdf#figure.3.4) of VMLS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5-element Array{Any,1}:\n",
       " \"char,avg, std\"              \n",
       " (\"a\", 1.0, 0.0)              \n",
       " (\"b\", 2.3, 2.41039415863879) \n",
       " (\"c\", 2.6, 3.773592452822641)\n",
       " (\"d\", 1.2, 4.308131845707603)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = ones(10);\n",
    "b = [ 5, 1, -2, 3, 6, 3, -1, 3, 4, 1 ];\n",
    "c = [ 5, 7, -2, 2, -3, 1, -1, 2, 7, 8 ];\n",
    "d = [ -1, -3, -4, -3, 7, -1, 0, 3, 9, 5 ];\n",
    "\n",
    "[(\"char,avg, std\"),\n",
    "(\"a\", avg(a), stdev(a)), \n",
    "(\"b\", avg(b), stdev(b)),\n",
    "(\"c\", avg(c), stdev(c)),\n",
    "(\"d\", avg(d), stdev(d))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Standardizing a vector.** If a vector $x$ isn’t constant (i.e., at least two of its entries are different), we can standardize it, by subtracting its mean and dividing by its standard deviation. The resulting standardized vector has mean value zero and RMS value one. Its entries are called $z-scores$. We’ll define a standardize function, and then check it with a random vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "standardize (generic function with 1 method)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function standardize(x)\n",
    "x_tilde = x .- avg(x) # De-meaned vector\n",
    "return x_tilde/rms(x_tilde)\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{Tuple{String,Float64,Float64},1}:\n",
       " (\"x:\", 0.4865676237006935, 0.5741240162744363)\n",
       " (\"z:\", 8.43769498715119e-17, 1.0)             "
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = rand(100);\n",
    "z = standardize(x);\n",
    "\n",
    "[(\"x:\", avg(x), rms(x)), (\"z:\", avg(z), rms(z))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mean or average value of the standarized vector $z$ is very nearly zero."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Angle\n",
    "**Angle.** Let’s define a function that computes the angle between two vectors. We will call it ang because Julia already includes a function angle (for the phase angle of a complex number)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{Tuple{Float64,String},1}:\n",
       " (0.9689825515916383, \":angle in radians\")\n",
       " (55.51861062801842, \":angle in degrees\") "
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define angle function, which returns radians\n",
    "ang(x,y) = acos(x'*y/(norm(x)*norm(y)));\n",
    "a = [1,2,-1]; b=[2,0,-3];\n",
    "\n",
    "[(ang(a,b),\":angle in radians\"), \n",
    "(ang(a,b)*(360/(2*pi)), \":angle in degrees\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Correlation coefficient.** The correlation coefficient between two vectors $a$ and $b$ (with nonzero standard deviation) is defined as $$ρ = \\frac{ã^Tb̃}{‖ã‖‖b̃‖},$$ where $ã$ and $b̃$ are the de-meaned versions of $a$ and $b$, respectively. There is no built-in function for correlation, so we can define one. We use function to calculate the correlation coefficients of the three pairs of vectors in Figure [3.8](https://web.stanford.edu/~boyd/vmls/vmls.pdf#figure.3.8) in VMLS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "correl_coef (generic function with 1 method)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function correl_coef(a,b)\n",
    "a_tilde = a .- avg(a)\n",
    "b_tilde = b .- avg(b)\n",
    "return (a_tilde'*b_tilde)/(norm(a_tilde)*norm(b_tilde))\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "a0 = [4.4, 9.4, 15.4, 12.4, 10.4, 1.4, -4.6, -5.6, -0.6, 7.4];\n",
    "b0 = [6.2, 11.2, 14.2, 14.2, 8.2, 2.2, -3.8, -4.8, -1.8, 4.2];\n",
    "a1 = [4.1, 10.1, 15.1, 13.1, 7.1, 2.1, -2.9, -5.9, 0.1, 7.1];\n",
    "b1 = [5.5, -0.5, -4.5, -3.5, 1.5, 7.5, 13.5, 14.5, 11.5, 4.5];\n",
    "a2 = [-5.0, 0.0, 5.0, 8.0, 13.0, 11.0, 1.0, 6.0, 4.0, 7.0];\n",
    "b2 = [5.8, 0.8, 7.8, 9.8, 0.8, 11.8, 10.8, 5.8, -0.2, -3.2];"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Tuple{String,Float64},1}:\n",
       " (\"0\", 0.9678196342570434)  \n",
       " (\"1\", -0.9875211120643734) \n",
       " (\"2\", 0.004020976661367021)"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(\"0\",correl_coef(a0,b0)),\n",
    "(\"1\",correl_coef(a1,b1)),\n",
    "(\"2\",correl_coef(a2,b2))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The correlation coefficients of the three pairs of vectors are $96.8\\%, −98.8\\%$, and $0.4\\%$.\n",
    "\n",
    "### 3.5 Complexity \n",
    "Let’s check that the time to compute the correlation coefficient of two $n$-vectors is approximately linear in $n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  0.037588 seconds (9 allocations: 15.259 MiB, 74.27% gc time)\n",
      "  0.422072 seconds (510.75 k allocations: 56.511 MiB, 30.05% gc time)\n",
      "  0.118583 seconds (9 allocations: 152.588 MiB, 19.66% gc time)\n",
      "  0.364146 seconds (463 allocations: 305.192 MiB, 36.15% gc time)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "2-element Array{Tuple{String,Tuple{Float64,Float64}},1}:\n",
       " (\"time for 0:\", (-0.002498368871007074, -0.002498368871007074))  \n",
       " (\"time for 1:\", (0.00023210986927802904, 0.00023210986927802904))"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x0 = randn(10^6); y0 = randn(10^6);\n",
    "x1 = randn(10^7); y1 = randn(10^7);\n",
    "\n",
    "[(\"time for 0:\", @time correl_coef(x0,y0), \n",
    "@time correl_coef(x0,y0)), \n",
    "(\"time for 1:\", @time correl_coef(x1,y1), \n",
    "@time correl_coef(x1,y1))]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.1.1",
   "language": "julia",
   "name": "julia-1.1"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.1.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
